Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🏃 Make client.MatchingLabels faster #2529

Merged
merged 1 commit into from
Oct 9, 2023
Merged

🏃 Make client.MatchingLabels faster #2529

merged 1 commit into from
Oct 9, 2023

Conversation

alvaroaleman
Copy link
Member

The 99% use-case of this is to set a selector, not to adjust an existing one. This change introduces a fastpath that does that with half the allocations and in a bit less than half the time.

The reason slowpath is slow is that for each label a requirement has to be constructed that is then appended to a slice, both of which cause allocations.

Fixes #2522

@timflannagan can you confirm that this fixes your issue? It brings the performance for the case that the labelselector is only set through MatchingLabels back to what it originally was.

The 99% use-case of this is to set a selector, not to adjust an existing
one. This change introduces a fastpath that does that with half the
allocations and in a bit less than half the time.

The reason slowpath is slow is that for each label a requirement has to
be constructed that is then appended to a slice, both of which cause
allocations.
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Oct 9, 2023
@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Oct 9, 2023
@alvaroaleman
Copy link
Member Author

I also have to say that while yes, this got slower, I find it surprising this is noticeable relative to DeepCopying in the cache or doing an api request. Would you mind describing what exactly you do that this becomes noticeable?

@timflannagan
Copy link

Awesome - I'll add a TODO to test this in our environment tomorrow/Tuesday. At a glance, I think this should help alleviate some of the hot paths we were seeing in our pprof profiles for our controller. I'd love to share them to help paint a better picture, but they're from a closed source product, and I don't want legal knocking down my door 😅

@alvaroaleman
Copy link
Member Author

@timflannagan right I figured that, but could you share some more details, so you seem to be doing a high number of List requests with client.MatchingLabels - What is the aproximate number of labels, do you list against the cache or the api, do you set any other listOptions?

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 9, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alvaroaleman, vincepri

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [alvaroaleman,vincepri]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 8ac8b9f into main Oct 9, 2023
9 checks passed
@timflannagan
Copy link

@alvaroaleman Just ran this commit through a couple of our heavier environments and these changes look like it fixes the performance issues we were previously seeing with 0.15.1 tag. Thanks for jumping on this (and the quick turn around). Any chance this could get backported to the release-0.15 release branch?

@samuelvl
Copy link

samuelvl commented Dec 8, 2023

/cherrypick release-0.16

@k8s-infra-cherrypick-robot

@samuelvl: only kubernetes-sigs org members may request cherry picks. You can still do the cherry-pick manually.

In response to this:

/cherrypick release-0.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@sbueringer
Copy link
Member

/cherry-pick release-0.15

@k8s-infra-cherrypick-robot

@sbueringer: new pull request created: #2614

In response to this:

/cherry-pick release-0.15

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Possible performance regression when upgrading to 0.15.2
7 participants